Papers by Miryam De Lhoneux
Supervised and Unsupervised Probing of Shortcut Learning: Case Study on the Emergence and Evolution of Syntactic Heuristics in BERT (2025.findings-acl)
Copied to clipboard
| Challenge: | Contemporary language models (LMs) rely on shortcut learning, using superficial cues that are spuriously correlated with labels. |
| Approach: | They propose to use syntactic heuristics to learn shortcuts in BERT when performing a task in Natural Language Understanding to investigate where these shortcuts emerge, how they evolve and how they impact the latent knowledge of the LM. |
| Outcome: | The proposed model rely on syntactic heuristics when performing a task in Natural Language Understanding. |
GRaMPa: Subword Regularisation by Skewing Uniform Segmentation Distributions with an Efficient Path-counting Markov Model (2025.acl-long)
Copied to clipboard
| Challenge: | Subword regularisations are known to be stochastic, but only a handful of possible segmentations are sampled. |
| Approach: | They propose to randomise word segmentations from a subword tokeniser instead of randomising them by weighting paths in an unweighted segmentation graph. |
| Outcome: | The proposed method outperforms existing methods on token-level tasks with spelling errors. |
Pixology: Probing the Linguistic and Visual Capabilities of Pixel-based Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | PIXEL is a vision transformer that has been pre-trained on rendered text . however, it is not able to outperform monolingual subwords like BERT . |
| Approach: | They propose to use PIXEL as a vision transformer to train on rendered text to explore the gap between its visual and linguistic understanding. |
| Outcome: | The proposed model outperforms monolingual subword models in most other contexts, but it lacks the linguistic knowledge to perform in language tasks. |
What is ”Typological Diversity” in NLP? (2024.emnlp-main)
Copied to clipboard
| Challenge: | linguistic typology is commonly used to motivate language selections, but there are no set definitions or criteria for such claims. |
| Approach: | They propose to use linguistic typology to motivate language selections on the basis that a broad typological sample ought to imply generalization across a wide range of languages. |
| Outcome: | The proposed measures show that skewed language selection can lead to overestimated multilingual performance. |